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GUIDANCE SYSTEM AND METHOD FOR SURGICAL PROCEDURES WITH 

IMPROVED FEEDBACK 

CROSS-REFERENCE TO RELATED APPLICATIONS 

[0001] The present patent application is a continuation-in-part of co-pending 

U.S. Patent Application, Serial No. 10/384,078, entitled "System and Method for Using a 
Haptic Device as an Input Device", filed on March 6, 2003, which claims the benefit of 
Provisional Patent Application, Serial No. 60/362,368, entitled "Haptic System for Medical 
Procedures", filed on March 6, 2002; the disclosures of both of which are incorporated herein 
by reference. This patent application is related to Patent Application Serial No. 10/384,077, 
entitled "System And Method For Intra-Operative Haptic Planning Of A Medical Procedure"; 
Patent Application Serial No. 10/384,072, entitled "System And Method For Haptic 
Sculpting Of Physical Objects"; and Patent Application Serial No. 10/384,194, entitled 
"System And Method For Interactive Haptic Positioning Of A Medical Device"; the 
disclosures of all of which are incorporated herein by reference. 

TECHNICAL FIELD OF THE INVENTION 

[0002] The present invention relates generally to computer-assisted surgery 

systems and surgical navigation systems, and more particularly to a system and method for 
conveying depth information during a medical procedure. 

BACKGROUND OF THE INVENTION 

[0003] The functions of a computer-assisted surgery (CAS) system may 

include pre-operative planning of a procedure, presenting pre-operative diagnostic 
information and images in useful formats, presenting status information about a procedure as 
it takes place, and enhancing performance. The CAS system may be used for procedures in 
traditional operating rooms, interventional radiology suites, mobile operating rooms or 
outpatient clinics. Many approaches to CAS have been attempted commercially. The 
procedure may be any medical procedure, whether surgical or non-surgical. 
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[0004] Navigation systems are used to display the positions of surgical tools 

with respect to pre- or intraoperative image datasets. These images include intraoperative 
images, such as two-dimensional fluoroscopic images, and preoperative three dimensional 
images generated using, for example, magnetic resonance imaging (MRI), computer 
tomography (CT) and positron emission tomography (PET). The most popular navigation 
systems make use of a tracking or localizing system. These systems locate markers attached 
or fixed to an object, such as an instrument or a patient, and track the position of markers. 
These tracking systems are optical and magnetic, but also include acoustic systems. Optical 
systems have a stationary stereo camera pair that observes passive reflective markers or 
active infrared LEDs attached to the tracked tools. Magnetic systems have a stationary field 
generator that emits a magnetic field that is sensed by small coils integrated into the tracked 
tools. These systems are sensitive to nearby metal objects. 

[0005] While navigation systems are relatively easy to integrate into the 

operating room, a fimdamental limitation is that they have restricted means of conunimication 
with the surgeon. Most systems transmit information to the surgeon via a computer monitor. 
Conversely, the surgeon transmits information to the system via a keyboard and mouse, 
touchscreen, voice commands, control pendant, or foot pedals, and also by moving the 
tracked tool. The visual displays of navigation systems may at best display multiple slices 
through three-dimensional diagnostic image datasets, which are not easy to interpret for 
complex 3-D geometries. These displays also require the surgeon to focus his visual attention 
away fi-om the surgical field. 

[0006] When defining a plan using a tracked tool, it can be difficult to 

simultaneously position the tool appropriately in multiple degrees of freedom (DOFs). 
Similarly, when aligning a tracked instrument with a plan, it is difficult to control the position 
of the tool in multiple simultaneous DOFs, especially where high-accuracy is desirable. It is 
perhaps not a coincidence that navigation systems have had their largest acceptance in cranial 
neurosurgery, where most applications involve specifying a trajectory to a feature of interest 
without hitting critical features. Often, the tip of the tool is pressed against the anatomy and 
pivoted, effectively decoupling the position and orientation planning of the trajectory. 

[0007] Autonomous robots have been applied commercially to joint 

replacement procedures. These systems make precise bone resections, improving implant fit 
and placement relative to techniques that rely on manual instruments. Registration is 
performed by having the robot touch fiducial markers screwed into the bones or a series of 
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points on the bone surfaces. Cutting is performed autonomously with a high-speed burr, 
although the surgeon can monitor progress and interrupt it if necessary. Bones must be 
clamped in place during registration and cutting, and are monitored for motion, which then 
requires re-registration. Deficiencies reported by users of these systems include the large size 
of the robot, poor ergonomics, the need for rigidly clamping the bone for the 45-60 minutes 
required for registration and cutting, and the need for increasing the incision by 50-100 mm 
to provide adequate access for the robot. Furthermore, autonomous robots generally function 
best in highly structured environments, as evidenced by the rigid clamping of the bones of 
interest and making larger incisions to keep soft tissue away from the robot. 

[0008] Except for specific steps of some surgical procedures, modem 

surgeries do not tend to provide well-structured environments for autonomous robots. A robot 
is generally not able to keep track of the surgical staff and instrumentation required to support 
a procedure. Although strict management of the operating environment might make this 
possible, the complexity of the human body will always provide a high degree of 
unstructuredness. 

[0009] Robotic technology can also be used to improve upon standard practice 

without requiring autonomous operation. Notable commercial systems of this type include 
teleoperated robotic systems for laproscopic surgeries ranging from gall-bladder removal to 
closed-chest beating heart coronary surgery. These systems provide a console for the surgeon 
that includes a high-fidelity display and a master input device. The slave robot is coupled to 
the master and physically interacts with the anatomy. The benefits of these systems are 
primarily in providing an ergonomic working environment for the surgeon while improving 
dexterity through motion scaling and tremor reduction. Although the master console would 
normally be in the same room as the patient, an interesting byproduct of these systems is that 
they enable telesurgery. However, the robots have minimal autonomy in these systems, which 
is not surprising given the complexity involved in manipulating and altering soft tissue. 

SUMMARY OF THE INVENTION 

[0010] It is often desirable to define objects with respect to images of an 

anatomy displayed using an image guided surgery system. For non-trivial objects, or those 
with complicated two or three dimensional forms, it may be difficult to present information in 
a manner that is simple for a user to imderstand. The local distance to a surface of interest, 



ATTORNEY'S DOCKET 
4204.38-1 



PATENT APPLICATION 



such as the surface of the defined object, or to a desired position, the local penetration 
distance of the surface of interest, or haptic repulsion force, often provides the most useful 
information for augmenting the interaction of the user with the image guided surgery system. 
The scalar value of the local distance may be conveyed to the user by visual, audio, tactile, 
haptic, or other means. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0011] For a more complete understanding of the present invention, the 

objects and advantages thereof, reference is now made to the following descriptions taken in 
connection with the accompanying drawings in which: 

[0012] FIGURE 1 is a diagrammatic illustration of an exemplary operating 

room in which a haptic device is used with a computer-assisted surgery system; 

[0013] FIGURE 2 illustrates an exemplary haptic device being used in 

conjunction with a computer-assisted surgery system; 

[0014] FIGURES 3 A and 3B illustrate different types of haptic objects; 

[0015] FIGURE 3C is a flowchart of an exemplary method for intra-operative 

haptic planning of a surgical procedure; 

[0016] FIGURE 4A illustrates the use of a dynamic haptic object for 

placement of a haptic device; 

[0017] FIGURE 4B is a flowchart of a method for interactive haptic 

positioning of a medical device coupled to a haptic device; 

[0018] FIGURE 5 illustrates the use of an exemplary haptic device in 

conjunction with a computer-assisted surgery system; 

[0019] FIGURE 6A illustrates an exemplary haptic device being used for 

haptic sculpting of physical objects; 

[0020] FIGURE 6B illustrates an exemplary haptic object for haptic sculpting 

of physical objects; 

[0021] FIGURE 6C is a flowchart of a method for dynamically modifying a 

haptic object; 

[0022] FIGURES 7A and 7B illustrate the use of an exemplary haptic device 

and a surgical tool to deflne a haptic object; 
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[0023] FIGURE 8 illustrates the use of an exemplary haptic device as an input 

device; 

[0024] FIGURE 9 is a flowchart of a representative method for using a haptic 

device as an input device; 

[0025] FIGURE 10 illustrates a system for conveying depth information 

during a medical procedure; and 

[0026] FIGURE 11 is a flowchart of a method for conveying depth 

information during a medical procedure. 

DETAILED DESCRIPTION OF THE DRAWINGS 

[0027] In the following description, lilce numerals refer to like elements. 

References to "surgeon" include any user of a computer-assisted surgical system, a surgeon 
being typically a primary user. References to "surgical procedure" include any medical 
procedure, whether interventional or non-interventional, an interventional procedure being 
typically the primary procedure. 

[0028] A haptic device is a mechanical or electro-mechanical device that 

interacts and communicates with a user, such as a surgeon, using sensory information such as 
touch, force, velocity, position, and/or torque. Some robots may be used as haptic devices, 
though haptic devices may include devices that are not necessarily considered to be robots in 
a conventional sense. Haptic devices typically have little autonomy. 

[0029] In general, a component of interest may be optionally coupled to the 

haptic devices. A component of interest may comprise a medical device, for example a 
surgical tool, a microscope, a laser range finder, a camera, a surgical light, an endoscope, an 
ultrasound probe, a radiotherapy device, interventional medical tools, rehabilitative systems 
for physical therapy, and/or the like. The terms "medical device", "surgical device" and 
"surgical tool" are used interchangeably herein. 

[0030] For example, when used during surgery, such devices cooperatively 

hold a surgical instrument in conjunction with the surgeon. The surgeon moves the surgical 
instrument with the assistance of, or input from, the haptic device. Alternatively, in a 
teleoperation system, the haptic device may exclusively hold the surgical instrument. In such 
an implementation, the surgeon moves a "master" haptic device that is coupled to a "slave" 
device in order to interactively manipulate the surgical tool. In a teleoperation system, the 
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master haptic device may be physically separated from the surgical site to provide a more 
ergonomic or immersive working position for the surgeon and/or allow the surgeon to 
perform the surgery remotely. In an impedance mode, a haptic device measures or senses the 
pose (position, orientation, velocity, and/or acceleration) of the surgical instrument and 
applies forces and/or torques ("wrench") to the instrument. In an "admittance" mode, a 
haptic device measures the wrench at some location on the device (or surgical instrument) 
and acts to modify the position of the instrument. There may be a static, quasi-static, or 
dynamic mapping between the sensed pose and output wrench. Common mappings may 
include wrenches that result from the tool interacting with "virtual" objects defined by or 
with input from a user, which may include mathematical or simulated mechanical constraints. 

[0031] A "haptic object" is used herein to describe such a mapping. In some 

cases, a haptic object may only produce non-zero outputs for certain joint angles of the haptic 
device, or only for certain endpoint positions and/or orientations of the haptic device. A 
haptic object may be a smoothly time varying mapping and/or may only exist for certain 
times. A haptic object may have an associated spatial or geometric representation that 
corresponds to locations where the mapping is discontinuous or has other properties that can 
be felt by the user when interacting with the haptic object. For example, if a haptic object 
only produces non-zero outputs when the endpoint of the haptic device lies within a spherical 
region in space, then it may be useful to present a corresponding spherical representation to 
the user. However, a haptic object may not necessarily have such a clearly defined boundary 
or similar internal structures. A haptic object may be active over the entire range of endpoint 
positions, endpoint orientations, and/or joint positions of the haptic device or only a portion 
of these ranges. There may be multiple haptic objects active at any given time, possibly in 
overlapping portions of space. 

[0032] A "haptic cue" is used to describe an aspect of the mapping of a haptic 

object. Having a cue may convey information or produce a desired effect when the user 
interacts with the haptic object. Haptic cues and haptic objects do not necessarily correspond 
to user interface or software programming components in a particular embodiment and may 
be simply one of many ways to design, implement, present to the user the mappings between 
the inputs and outputs of the haptic device. 

[0033] The reduction or elimination of autonomy increases the comfort level 

of users, such as surgeons. Any time a robot moves autonomously, the surgeon is no longer in 
control and must simply observe the robot's progress. Robot motions have to be slow to 



ATTORNEY'S DOCKET 
4204.38-1 



PATENT APPLICATION 



provide adequate time for the surgeon to respond should something unusual happen. If, 
however, a robot acts, at least mostly, in a passive manner, even if capable of active motions, 
then the surgeon does not cede control to the robot. 

[0034] Using a device capable of active motions in such a way that it only acts 

like a passive device from the user's perspective has advantages. Active actuators can be 
used to counteract the effect of gravity, allowing a greater variety of mechanism designs. 
The device can be used in an autonomous mode for performing automated testing and service 
procedures. 

[0035] FIGURE 1 is a diagrammatic illustration of an exemplary operating 

room in which a haptic device 113 is used with a computer-assisted surgery system 11. 
Computer-assisted surgery system 1 1 comprises a display device 30, an input device 34, and 
a processor based system 36, for example a computer. Input device 34 may be any input 
device now known or later developed, for example, a keyboard, a mouse, a trackball, and/or 
the like. Display device 30 may be any display device now known or later developed for 
displaying two-dimensional and/or three-dimensional images, for example a monitor, a 
wearable display, a projection display, a head-mounted display, stereoscopic views, a display 
device capable of displaying image(s) projected from an image projecting device, for 
example a projector, and/or the like. If desired, display device 30 may be a display device 
capable of displaying a holographic image. If desired, display device 30 may be a touch 
screen and be used as an input device. 

[0036] Haptic device 113 is, in the illustrated example, a robotic device. 

Haptic device 113 may be controlled by a processor based system, for example a computer 
10. Computer 20 may also include power amplification and input/output hardware. Haptic 
device 113 may communicate with computer-assisted surgery system 11 by any 
commimication mechanism now known or later developed, whether wired or wireless. 

[0037] Also shown in FIGURE 1 is a storage medium 12 coupled to processor 

based system 36. Storage medium 12 may accept a digital medium which stores software 
and/or other data. A surgical tool or instrument 1 12 is shown coupled to haptic device 113. 
Surgical tool 112 is preferably mechanically coupled to haptic device 113, such as by 
attaching or fastening it. However, if desired, surgical tool 112 may be coupled, either 
directly or indirectly, to haptic device 1 13 by any other method, for example magnetically. If 
desired, vacuum may be used to couple surgical tool 1 12 to haptic device 113. Surgical tool 



ATTORNEY'S DOCKET 
4204.38-1 



PATENT APPLICATION 



1 12 may be haptically controlled by a surgeon remotely or haptically controlled by a surgeon 
116 present in proximity to surgical tool 112, 

[0038] Haptic object 1 10 is a virtual object used to guide and/or constrain the 

movement and operations of surgical tool 1 12 to a target area inside a patient's anatomy 1 14, 
for example the patient's leg. In this example, haptic object 110 is used to aid the surgeon to 
target and approach the intended anatomical site of the patient. Haptic feedback forces are 
used to slow and/or stop the surgical tool's movement if it is detected that a portion of 
surgical tool 112 will intrude or cross over predefined boimdaries of the haptic object. 
Furthermore, haptic feedback forces can also be used to attract (or repulse) surgical tool 1 12 
toward (or away from) haptic object 1 10 and to (or away from) the target. If desired, surgeon 
116 may be presented with a representation of the anatomy being operated on and/or a virtual 
representation of surgical tool 1 12 and/or haptic object 1 10 on display 30. 

[0039] When surgical tool 1 12 is haptically controlled by a surgeon remotely, 

for example when conducting a teleoperation, the surgeon controls the movement of the 
surgical tool using the master haptic device and/or a real or simulated display of the surgical 
tool, patient anatomy, and/or additional haptic or visual objects designed to aid the surgical 
procedure. Haptic feedback forces may be transmitted by slave haptic device 113 to the 
surgeon at the remote location via the master haptic device to guide the surgeon. 
Alternatively, the haptic feedback forces may be generated at the master device and 
transmitted to the surgeon directly. In some cases either the slave or master device may be a 
positioning device with little or no haptic capabilities. 

[0040] The CAS system preferably includes a localization or tracking system 

that determines or tracks the position and/or orientation of various trackable objects, such as 
surgical instruments, tools, haptic devices, patients, and/or the like. The tracking system 
continuously determines, or tracks, the position of one or more trackable markers disposed 
on, incorporated into, or inherently a part of the trackable objects, with respect to a three- 
dimensional coordinate ft^me of reference. Markers can take several forms, including those 
that can be located using optical (or visual), magnetic or acoustical methods. Furthermore, at 
least in the case of optical or visual systems, location of an object's position may be based on 
intrinsic features, landmarks, shape, color, or other visual appearances, that, in effect, 
function as recognizable markers. 

[0041] Any type of tracking system may be used, including optical, magnetic, 

and/or acoustic systems, that may or may not rely on markers. Present day tracking systems 
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are typically optical, functioning primarily in the infrared range. They usually include a 
stationary stereo camera pair that is focused around the area of interest and sensitive to 
infrared radiation. Markers emit infrared radiation, either actively or passively. An example 
of an active marker is a light emitting diodes (LEDs). An example of a passive marker is a 
reflective marker, such as ball-shaped marker with a surface that reflects incident infrared 
radiation. Passive systems require a an infrared radiation source to illuminate the area of 
focus. A magnetic system may have a stationary field generator that emits a magnetic field 
that is sensed by small coils integrated into the tracked tools. 

[0042] With information from the tracking system on the location of the 

trackable markers, CAS system 11 is progranmied to be able to determine the three- 
dimensional coordinates of an end point or tip of a tool and, optionally, its primary axis using 
predefined or known (e.g. from calibration) geometrical relationships between trackable 
markers on the tool and the end point and/or axis of the tool. A patient, or portions of the 
patient's anatomy, can also be tracked by attachment of arrays of trackable markers. In the 
illustrated example, the localizer is an optical tracking system that comprises one or more 
cameras 14 that preferably track a probe 16. As shown in FIGURE 1, cameras 14 may be 
coupled to processor based system 36. If desired, cameras 14 may be coupled to computer 
10. Probe 16 may be a conventional probe now known or later developed. If desired, the 
probe may be rigidly attached to haptic device 113 or integrated into the design of haptic 
device 113. 

[0043] If desired, in an implementation, processor based system 36 may 

comprise a portion of image guided surgery software to provide minimal user functionality 
e.g., retrieval of previously saved surgical information, preoperative surgical planning, 
determining the position of the tip and axis of instruments, registering a patient and 
preoperative and/or intraoperative diagnostic image datasets to the coordinate system of the 
tracking system, etc. Image guided surgery using this method may not be possible with the 
computer alone. As such, fiill user functionality may be enabled by providing the proper 
digital medium to storage medium 12 coupled to computer 36. The digital medium may 
comprise an application specific software module. The digital medium may also comprise 
descriptive information concerning the surgical tools and other accessories. The application 
specific software module may be used to assist a surgeon with planning and/or navigation 
during specific types of procedures. For example, the software module may display 
predefined pages or images corresponding to specific steps or stages of a surgical procedure. 
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At a particular stage or part of a module, a surgeon may be automatically prompted to 
perform certain tasks or to define or enter specific data that will permit, for example, the 
module to determine and display appropriate placement and alignment of instrumentation or 
implants or provide feedback to the surgeon. Other pages may be set up to display diagnostic 
images for navigation and to provide certain data that is calculated by the system for 
feedback to the surgeon. Instead of or in addition to using visual means, the CAS system 
could also communicate information in ways, including using audibly (e.g. using voice 
synthesis) and tactilely, such as by using a haptic interface of device. For example, in 
addition to indicating visually a trajectory for a drill or saw on the screen, a CAS system may 
feedback to a surgeon information whether he is nearing some object or is on coiirse with an 
audible sound. To fiirther reduce the burden on the surgeon, the module may automatically 
detect the stage of the procedure by recognizing the instrument picked up by a surgeon and 
move immediately to the part of the program in which that tool is used. 

[0044] The software module may be such that it can only be used a predefined 

number of times. If desired, the software module fimctions only when used in conjunction 
with the portion of the image guided surgery software that resides on computer 36. The 
software which resides on computer 36 in conjunction with the software on the digital 
medium processes electronic medical diagnostic images, registers the acquired images to the 
patient's anatomy, and/or registers the acquired images to any other acquired imaging 
modalities, e.g., fluoroscopy to CT, MRI, etc. if desired, the image datasets may be time 
variant, i.e. image datasets taken at different times may be used. Media storing the software 
module can be sold bundled with disposable instruments specifically intended for the 
procedure. Thus, the software module need not be distributed with the CAS system. 
Furthermore, the software module can be designed to work with specific tools and implants 
and distributed with those tools and implants. Moreover, CAS system can be used in some 
procedures without the diagnostic image datasets, with only the patient being registered. 
Thus, the CAS system need not support the use of diagnostic images in some applications - 
i.e. an imageless application. 

[0045] An example of the illustrated robotic arm is a robotic arm 

manufactured by Barrett Technology, and referred to as the "Whole-Arm Manipulator" or 
"WAM". This robotic arm has a cable transmission, which provides high bandwidth, 
backdrivability, and force fidelity. However, other robotic devices capable of impedance or 
admittance modes of haptic interaction could be used. For example, direct-drive systems or 



ATTORNEY'S DOCKET 
4204.38-1 



PATENT APPLICATION 



systems with other types of low-friction transmissions or systems with a combination of 
transmission types may also be well-suited to serve as a haptic device for surgical 
applications. Furthermore, the haptic device need not necessarily take the form of a robotic 
arm. The WAM robotic arm has a four degrees of freedom of movement. However, it is 
augmented by a 1-DOF direct-drive wrist for trajectory-based medical applications. If 
desired, degrees of freedom may be added or removed without affecting the scope of the 
illustrated invention. 

[0046] Though it has some advantages, a cable transmission has some 

disadvantages. It requires careful installation and maintenance to prevent the possibility of 
failure during a procedure. Furthermore, a cable transmission is not as stiff as geared 
transmissions. Similar deficiencies may also be found in haptic devices using other types of 
transmissions. 

[0047] These deficiencies may be addressed by augmenting existing position 

sensors that are mounted on drive motors with additional redundant sensors. These sensors 
may be of various types, including without limitation rotary encoders or resolvers, tilt 
sensors, heading (compass) sensors, sensors that detect the direction of gravity, an optical, 
magnetic or acoustical tracking system (such as optical camera systems of the type 
commonly used to track surgical instruments), or laser-based position sensing. The output of 
these sensors can be compared with the original sensors to detect discrepancies that may 
indicate problems in the transmissions or sensors. In addition, the added sensors can be used 
to detect both low bandwidth deflections in the cable transmissions, which the system can 
then easily compensate for using well-known control techniques. The sensor may also detect 
the high bandwidth deflections in the cable transmissions, which can provide an additional 
input to the servo loop and permit improved stability of the servo system, using well-known 
control techniques for systems that include sensors on both the drive and load sides of a 
transmission. The sensor can also improve the accuracy of the determination of the pose of 
the arm by reducing or eliminating the effect of deflections of the arm links and/or 
transmission. Such sensors could also be used to overcome similar deficiencies in robotic 
devices using other types of transmission systems. 

[0048] When performing surgery, a haptic device capable of holding a tool, 

e.g. a drill guide or other similar constraint or attachment mechanism for surgical tools is 
positioned relative to the patient such that it can attain the poses appropriate for a variety of 
approaches for a particular procedure. It is also registered to the physical anatomy such that it 
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can correlate information in diagnostic or planning image datasets, which can be two or three 
dimensional, to locations in physical space using well-known registration techniques. The 
image datasets may be one or more images generated using for example, magnetic resonance 
imaging (MRI), computer tomography (CT), positron emission tomography (PET), magnetic 
resonance angiography (MRA), single photon emission computed tomography (SPECT), 
magnetic resonance venography (MRV), contrast enhanced MR venography (CEMRV), CT 
angiography, CT myelography, MR angiography, MR myelography, fluoroscopy, optical 
imaging, isotope imaging, ultrasound microscopy, laproscopic ultrasound, and MR 
spectrometry. Such images may include, for example, x-ray images, digital x-ray images, 
computer tomography images, MRI images, MRA images, MR spectrometric images, PET 
images, MRV images, SPECT images, CEMRV images, CT angiographic images, CT 
myelographic images, MR myelographic images, flair images, two-dimensional fluoroscopic 
images, three-dimensional fluoroscopic images, two-dimensional ultrasonic images, three- 
dimensional ultrasonic images, ultrasound microscopy images, laproscopic ultrasound 
images, optical images, isotopic images, laser depth maps, line arts, sketches, "cartoon" 
representations, holographic images, and/or the like. 

[0049] Features to be avoided, such as blood vessels, tendons, nerves, and 

critical areas of the brain can be automatically, semi-automatically, or manually defined on 
the image datasets. Features targeted by the procedure, such as tumors, osteophytes, 
anatomical targets for deep-brain stimulation, biopsy sites, anatomical sites for implant 
placement, or other regions of the anatomy can also be automatically, semi-automatically, or 
manually defined on the image datasets. 

[0050] The image dataset(s), coupled with definitions of features to be 

avoided, can be used to create haptic "cues" that indicate to the surgeon that a violation of 
sensitive anatomy is taking place. A general fimction of these types of cues is to apply forces 
and/or torques that tend to repulse the haptic device from poses where an instmment attached 
to the device would, for example, impact the defined critical features. Similarly, the image 
dataset(s), coupled with the definitions of features to be targeted can also used to create 
haptic cues that indicate to the surgeon that the desired target region would be reached by the 
surgical instrument appropriately attached to the haptic arm. A general fiinction of these 
types of cues is to attract the haptic device to such poses or lock the haptic device into these 
poses once they are attained. 
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[0051] While the haptic device can be deployed as a fully integrated 

component of a computer-aided surgery system, there are advantages to having the haptic 
device act as an optional peripheral to such a system. The system is then convenient to use 
for procedures that do not require the use of the haptic device. There are also development 
and architectural advantages to this approach. The haptic device will likely require a real- 
time operating system or special motion control hardware to generate high-frequency updates 
for the haptic control system. The computer-aided surgery system will have different 
requirements, such as fast graphics processing hardware and compatibility requirements with 
a range of user input and output devices, so that there are advantages of having two computer 
systems to meet the differing uses. Separating the computer surgery and haptic arm 
components also has safety advantages. The haptic device therefore preferably contains only 
computing software and hardware that is necessary for ensuring high-performance, stable, 
and safe operation. The computer aided surgery system can contain software and hardware 
for connecting to a hospital network, displaying various graphical views, supporting various 
user input/output devices, managing libraries of implant and instrument databases, and/or any 
other functionality useful in such a system. This architecture also allows developers with 
minimal knowledge of haptic systems to build applications that use the haptic device. The 
physical interface between these two systems can be wired or wireless, such as a serial, USB, 
or other cable communications interface, or wireless ethemet, wireless serial, infra-red or 
other wireless communications system. The software interface between these systems would 
include a set of commands that allows the computer aided surgery system to control operation 
of the haptic device. For example, the computer-aided surgery system may send a command 
to the haptic device requesting it to enter into a joystick-like input mode with certain stiffness 
parameters. The haptic arm system checks if the parameters are safe and otherwise 
acceptable, and then enters into such a mode or responds with an appropriate error message. 
The computer-aided surgery system and haptic device may also be integrated into a single 
system unit, or may be implemented using a single or a multi-processor computing device. 
The CAS system, the haptic device and/or computer 10 may also be integrated into another 
piece of equipment, such as an imaging equipment (e.g., fluoroscopy, CT, MR, ultrasound, 
and/or the like), an equipment cart in the room where the medical procedure is performed, 
and/or the like. 

[0052] Referring to FIGURE 2, representative "haptic object" 20 is a two- 

dimensional virtual plane. However, it is only an example of haptic objects generally, which 
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may be zero (e.g. a point), one (e.g. a virtual line or path), two (e.g. a virtual plane or flat 
surface), or three dimensional (e.g. a virtual curved surface, a cube or other solid object), and 
may have simple or complex geometric shapes. Haptic object 20 is preferably defined with 
respect to the space of a physical object, such as patient anatomy 1 14. Haptic object 20 is 
defined to guide and/or constrain the movement of haptic device 113. The distance between 
haptic device 1 13 and haptic object 20 is shown in FIGURE 2 by X and the distance between 
patient's anatomy 1 14 and haptic object 20 is shown by Xi. Haptic object 20 may be used in 
connection with generating force feedback on haptic device 113. The generation of force 
feedback may also depend on various factors, for example, the velocity at which haptic 
device 113 is approaching patient's anatomy 114, the position of haptic device 113, haptic 
object 20, and/or the like. An algorithm which computes the current position of haptic device 
1 13 relative to haptic object 20 may be used to provide information to the sxirgeon about the 
location of haptic device 113 relative to haptic object 20. When haptic device 113 comes 
within a predefined distance of haptic object 20, a stiffness parameter may be changed to 
make it more difficult to move haptic device 113. If desired, force may be applied in a 
direction away fi-om anatomy 114 to resist the movement of haptic device 113 toward 
anatomy 1 14 or to move haptic device 113 away fi*om anatomy 1 14. 

[0053] It may not be appropriate to implement rigid haptic objects, such as 

virtual surfaces and walls, in certain cases. A surgeon will lose the ability to feel the anatomy 
in any direction that is rigidly constrained by the haptic device. In many applications, precise 
localization of anatomical features cannot be achieved by simply combining diagnostic 
datasets with a tool tracking system or precision robotic devices. Changes in the anatomy 
after the diagnostic datasets are taken, unsensed motion in the kinematic chain connecting the 
anatomical features of interest and the tracking system's camera or haptic device, registration 
errors, and inaccuracies in the localization devices will contribute to positioning errors. 
Although CAS systems may be used to position the surgical tool very close to the target 
region, more accurate positioning is often difficuU or prohibitively costly. In some medical 
procedures, such as pedicle screw placement in the upper thoracic and cervical portions of the 
spine, deep brain neurosurgical procedures, etc., a slight inaccuracy may adversely affect the 
medical procedure being performed. Therefore, it is desirable in these types of procedures 
that a surgeon retain an ability to feel the anatomy. 

[0054] Haptic devices can be used for registering patients to CAS systems and 

diagnostic data sets of the patient's anatomy, for example, by attaching a probe and touching 
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it to a few selected anatomical landmarks, implanted fiducials, or multiple points on a surface 
of interest. They can be used for haptic exploration of diagnostic datasets to augment the 
visual display of this information. This exploration may occur intra-operatively while 
registered to the actual patient anatomy or pre-operatively in a purely virtual way. This 
haptic exploration is especially useful for exploring complex three-dimensional structures, 
where the surgeon's highly developed sense of touch can be used to explore complexities or 
subtleties of the dataset that may be difficult or impossible to display adequately on a two- 
dimensional or even three-dimensional visual display. 

[0055] While performing traditional freehand surgery, surgeons rely on local 

anatomical features to ensure proper positioning of the surgical tool. If the ability of the 
surgeon to feel the patient anatomy is preserved, the surgeon can explore the local anatomy 
and correct these localization errors based on his expert knowledge of structures of interest. 
In this way, the final positioning is determined by nearby anatomical features rather than a 
tracking system sitting across the operating room or a robot whose base may not be rigidly 
connected to the patient. 

[0056] A portion of surgical tool 112 coupled with a haptic device, for 

example the tip of surgical tool 112, may be used to sense properties of the local anatomy. 
The properties of the local anatomy may be used to position surgical tool 1 12 or to verify the 
proper positioning of surgical tool 112. The properties that may be sensed or monitored by 
the tool include electrical properties of the anatomy, force, pressure, stiffness, conductivity, 
etc. The information from the tip may be provided back to CAS system 1 1. The information 
may then, if desired, be correlated with information from diagnostic image datasets of the 
patient. If desired, information from the tool may be used to augment or replace the 
information from the image datasets. In either case the information may be used for better 
placement of surgical tool 112. 

[0057] Location or position information of the tool may be sensed and 

provided back to CAS system 1 1 without the use of a separate sensor. The surgeon may 
manually move surgical tool 112 to the desired position. Position information of the tip of 
surgical tool 1 12 in the desired position may be determined directly by CAS system 1 1 and/or 
computer 10 without the use of a separate sensor. Other properties of the anatomy may be 
sensed by placing sensors at the tip of surgical tool 1 12. The output from the sensors may be 
provided back to CAS system 1 1 for processing. 
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[0058] The collected information may be used for a variety of purposes, such 

as alerting the user to registration errors, fully or partially correcting registration errors, 
displaying graphical representations of the information on display device 30, defining haptic 
objects to assist the user, displaying graphical representations of the information on display 
device 30 superimposed over one or more images of the anatomy, and/or the like. If desired, 
the collected information may be logged for use in machine learning techniques. 

[0059] The combination of a haptic device and a CAS system is also useful 

for combining haptic exploration of diagnostic datasets and use of the haptic device as a 
primary input device for planning. In this way, haptic exploration naturally leads the user to 
a suitable plan for performing a procedure. Additionally, in some circumstances it is possible 
to have the haptic device and the tool coupled with it in the correct position for performing a 
procedure as a result of this exploration/planning process, eliminating the need to move the 
haptic device into position as a separate step. 

[0060] Referring to FIGURE 3A, it may be desirable in certain procedures to 

confine the surgical instrument to a small working volume, in which case it may stay within a 
working area inside a haptic object during the entire procedure. It may be necessary in 
certain cases to segment or define manually certain important features, but for most 
applications automated segmentation of the diagnostic datasets will be sufficient for 
providing appropriate haptic feedback, 

[0061] In the illustrated embodiment, one or more attractive haptic objects are 

associated with a target region for performing the surgical procedure and one or more 
repulsive haptic objects are associated with anatomical features to be avoided during the 
surgical procedure. For example, as shown in FIGURE 3A, haptic object 22 defines a 
working area or volume for constraining movement of surgical tool 112. On the other hand, 
as shown in FIGURE 3B, haptic object 24 defines a working area or volume for constraining 
movement of surgical tool 112 so that it is prevented from coming close to critical regions, 
such as nerves 25, organs 27, etc. For example, once the haptic objects are defined, the user 
performs surgical planning by pushing haptic device 113 around until a pose is found where 
the cues from the attractive haptic objects are active indicating that surgical tool 112, when 
attached to haptic device 113, would reach the target region, and where the cues from the 
repulsive haptic objects are inactive, indicating that surgical tool 112 would not penetrate any 
of the defined sensitive anatomical regions. In most cases, these requirements will not fully 
constrain the pose of the arm and the user can move the arm within this range of acceptable 
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approaches based on any secondary criteria a user finds appropriate. In some cases, the arm 
may achieve an equilibrium state where multiple attractive or repulsive haptic cues act in 
opposite directions. The user might mistake this configuration to be an acceptable pose, even 
though the target region might not be reached or the critical anatomy regions might be 
violated. The user may be alerted to this situation in a number of ways, including audible or 
visual indicators, or by a haptic cue such as a vibration of haptic device 113. The user could 
then correct this situation by pushing the haptic device away from this pose. Once in a pose 
satisfactory to the user, haptic device 1 13 can be locked into position, using hardware brakes, 
control servoing techniques, or any other appropriate method to provide a stable physical 
reference for the surgical procedure. 

[0062] If fine adjustments are desired, the haptic device can be operated using 

a mode where motion scaling, constraints, or other methods are used to make such 
corrections that might otherwise be beyond the dexterity of the surgeon. For example, a 
control servo can be enabled to lock the device to a certain finite stiffness at the approximate 
desired pose. The surgeon can then make fine adjustments to this pose using a variety of 
methods. For example, the surgeon may use a touch screen, a keyboard, a mouse, a trackball 
or voice inputs. If desired, the surgeon may push the end of the haptic device in the desired 
direction. In response to these inputs, the system would adjust the desired pose appropriately, 
possibly in small increments that would be difficult to achieve by direct positioning of the 
haptic device. It may be desirable to lock only a portion of the pose so that the surgeon can 
focus on a more limited number of adjustments at one time. This fine adjustment may occur 
after the coarse haptic positioning is complete, simultaneous with the coarse haptic 
positioning, or interleaved with the coarse haptic positioning. 

[0063] For example, selecting a trajectory for a cranial neurosurgical 

procedure such as a biopsy, tumor resection, or deep-brain stimulation is a complicated 3-D 
planning problem. The surgeon must find a path to a target area while avoiding blood vessels 
and sensitive areas of the brain. If these regions can be turned into repulsive haptic objects, 
planning such a procedure may be as simple as applying a haptic constraint that keeps the 
trajectory of a tool guide passing through the target of interest, and allowing the user to pivot 
the device about this point until it settles into a suitable pose where none of the repulsive 
haptic objects are violated. 

[0064] FIGURE 3C is a flowchart of an exemplary method 140 for intra- 

operative haptic planning of a surgical procedure. Haptic device 113 is placed in the 
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operating room such that surgical tool 112 may be positioned over a large portion of a 
clinically reasonable range of surgical approaches for a given surgical procedure. Siu-gical 
planning using method 140 is performed in the presence of the patient and preferably without 
surgical tool 1 12 being coupled to haptic device 1 13. Surgical tool 1 12 may be a non-contact 
medical device, such as a diagnostic or therapeutic radiation source. If desired, surgical 
planning using method 140 may be performed with surgical tool 1 12 coupled to haptic device 
113 but being in a retracted state. When surgical tool 112 comprises a non-contact medical 
device, it is preferably in a disabled state. A representation of the anatomy of the patient to 
be operated on may be displayed on display device 30 along with a "virtual tool". The virtual 
tool may be a high-fidelity representation or a schematic representation of surgical tool 112, 
such as an axis, a point, or other feature of surgical tool 112. The virtual tool indicates 
relative to the anatomy of the patient, the position and/or angle of surgical tool 112 or some 
portion thereof if the surgical tool had been coupled to haptic device 113 in its normal or 
enabled state. 

[0065] In step 142, haptic device 113 is registered to the anatomy of the 

patient. If desired, the representation of the anatomy of the patient displayed on display 
device 30 may also be registered with the anatomy of the patient so that information in 
diagnostic or planning datasets may be correlated to locations in the physical space. Any 
method for registering, now known or later developed, may be used. In step 144, the target 
region is defined. The target region may be, for example, a tumor, an osteophyte, an 
anatomical target for deep-brain stimulation, a bone channel, and/or the like. The target 
region may be defined in any manner now known or later developed. For example, the user, 
such as the surgeon, may manually identify the target region on display device 30. If desired, 
the surgeon may define the target region by touching one or more points on the target region 
or circling the target region on display device 30 with a tool. Altematively, the surgeon may 
define the target region by pointing a tool mounting axis of haptic device 113 to the target 
region or by using haptic device 113 as an input device. Preferably, the identified target 
region is automatically highlighted on display device 30. The tool mounting axis of haptic 
device 113 may be of any shape, for example curved, straight, and/or the like. Regardless of 
the manner in which the target region is defined, it is desirable that once defined, the target 
region be clearly displayed on display device 30 for confirmation. One or more attractive 
haptic objects, such as haptic object 22 of FIGURE 3A, may be associated with the target 
region. 
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[00661 ^ step 146, anatomical obstacles to be avoided are defined. The 

anatomical obstacles comprise features to be avoided during surgery, such as major blood 
vessels, tendons, nerves, critical areas of the brain, organs, healthy bones or other tissues, 
and/or the like. The anatomical obstacles may be defined in any manner now known or later 
developed. For example, the surgeon may manually identify the anatomical obstacles on 
display device 30. If desired, the surgeon may define the anatomical obstacles by touching 
one or more points on the anatomical obstacles or circling the anatomical obstacles on display 
device 30 with a tool. Alternatively, the surgeon may define the anatomical obstacles by 
pointing the tool mounting axis of haptic device 113 to the anatomical obstacles or by using 
haptic device 113 as an input device. Preferably, the identified anatomical obstacles are 
highlighted on display device 30. Regardless of the manner in which the anatomical 
obstacles are defined, it is desirable that, once defined, the anatomical obstacles are clearly 
displayed on display device 30 for confirmation. One or more repulsive haptic objects, such 
as haptic object 24 of FIGURE 3B, may be associated with the defined anatomical obstacles. 
Preferably, each anatomical obstacle has one repulsive haptic object associated with it, 
although if desired more than one repulsive haptic object may be associated with an 
anatomical obstacle. 

[0067] In step 148, haptic device 1 13 is positioned, preferably by the surgeon, 

such that if surgical tool 1 12 were coupled to haptic device 1 13 or if surgical tool 1 12 were in 
an operating state, then the appropriate portion of the surgical tool would have the desired 
relationship with the target region. For example, when coupled to haptic device 113, surgical 
tool 112 would penetrate the target region. Surgical tool 1 12 is in its operating state when it 
is coupled to haptic device 113 and is not retracted and/or is not disabled. Step 148 is 
preferably performed without regard to whether or not the tool may intersect the anatomical 
obstacles in this position. A virtual tool displayed on display device 30 is such that it's 
position and orientation corresponds to the position and orientation of surgical tool 112 if 
surgical tool 112 had been mounted on haptic device 113 or if surgical tool 112 were in its 
normal operating state. Thus, the surgeon may position haptic device 1 13 in the desired pose 
while viewing the display on device 30, such that the virtual tool has the appropriate relation 
with the target region. 

[0068] In step 152, a determination is made as to whether the virtual tool is 

intersecting any anatomical obstacles. If the virtual tool is not intersecting any anatomical 
obstacles, then the process starting at step 162 is executed. Otherwise, the process starting at 
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step 154 is executed. In step 154, haptic cues are provided by haptic device 113 to the user. 
The haptic cues may be provided to the user based on one or more haptic objects, for example 
the attractive haptic object(s) associated with the target region and/or the repulsive haptic 
object(s) associated with the anatomical obstacles. The repulsive haptic object(s) generate 
forces and/or torques that guide haptic device 113 away from poses where the virtual tool 
would intersect the anatomical obstacles. Preferably, the repulsive haptic cues are active 
when the virtual tool penetrates the repulsive haptic objects or is in proximity to the repulsive 
haptic objects. The attractive haptic object(s) cause the haptic device to generate forces 
and/or torques that guide haptic device 113 toward poses where the virtual tool has the 
desired relationship with the target region. 

[0069] It is possible that the position of haptic device 113 may be such that 

cues from multiple haptic objects cancel each other out even though the virtual tool may be 
violating the anatomical obstacles. As such, in step 156, a determination is made as to 
whether haptic cues from muhiple obstacles are canceling each other out. If haptic cues from 
multiple obstacles are not canceling each other out, then the process starting at step 158 may 
be executed. If haptic cues from multiple obstacles are canceling each other out, then in step 
160, a special haptic cue, for example a vibration, may be provided to alert the user of this 
situation and the process starting at step 158 may be executed. 

[0070] In step 158, haptic device 113 is moved, preferably by the surgeon. 

Haptic device 113 is preferably moved based at least in part on the haptic cues provided by 
haptic device 113 to the surgeon. The position of surgical tool 112 had it been coupled to 
haptic device 113 is tracked by the virtual tool and displayed on display device 30. 
Preferably, the user moves haptic device 113 until an equilibrium pose is found. In the 
equilibrium position, the cues created by the attractive haptic objects are active and those 
created by the repulsive haptic objects are inactive. The process starting at step 152 may then 
be executed to determine whether the virtual tool is intersecting any anatomical obstacles. 

[0071] In step 162, a determination is made as to whether the user is satisfied 

with the trajectory to the target region. The user may make this determination by viewing the 
virtual tool relative to the target region as illustrated on display device 30. If the user is not 
satisfied with the position and/or the orientation of the virtual tool, then the process starting at 
step 158 may be executed. If the user is satisfied with the position and the orientation of the 
virtual tool relative to the target region and the obstacles, then the process starting at step 164 
may be executed. The user may indicate its satisfaction in one or more of a number of ways. 
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For example, the user may issue a voice command to indicate that it is satisfied with the 
position and orientation of the virtual tool. If desired, the user may activate a foot pedal or a 
button associated with the computer-assisted surgery system or haptic device 113 to indicate 
its satisfaction. If desired, the user may indicate its satisfaction via a touch screen, a 
keyboard, a mouse, and/or the like, associated with the computer-assisted surgery system or 
haptic device 113. In step 164, haptic device 1 13 may be locked in the current pose. 

[0072] Once the pose of haptic device 113 is locked, the surgical procedure 

may be performed, for example by coupling surgical tool 112 to haptic device 113 or by 
placing surgical tool 112 in its fully functional or operational configuration. Because the 
pose of surgical tool 112 relative to the anatomy has already been determined with the aid of 
the virtual tool, surgical tool 1 12 will achieve the desired position when it is coupled to haptic 
device 1 13 or when it is configured for use. 

[0073] The illustrated method for intra-operative haptic planning of a surgical 

procedure may be implemented in software, hardware, or a combination of both software and 
hardware. The steps discussed herein need not be performed in the stated order. Several of 
the steps could be performed concurrently with each other. Furthermore, if desired, one or 
more of the above described steps may be optional or may be combined without departing 
from the scope of the present invention. Furthermore, one or more of the above described 
steps may be performed outside the operating room to save time spent in the operating room. 
For example, steps 144 and 146 may be performed prior to bringing the patient into the 
operating room and prior to step 142,. 

[0074] A technical advantage of this exemplary embodiment for intra- 

operative haptic planning of a surgical procedure is that it provides for tighter coupling of the 
planning and execution phases of the surgical procedure. Planning for the surgical procedure 
is preferably performed intra-operatively with respect to the patient. Thus, when planning is 
complete, the haptic device is in position for executing the surgical plan. No additional 
motion of the haptic device is required to initiate the execution phase. Furthermore, by using 
a virtual tool to determine the trajectory of the real surgical tool to the target region, injury to 
anatomical features may be avoided during the planning phase. 

[0075] A haptic object may be of any shape or size. As shown in FIGURE 

4A, haptic object 26 may be funnel shaped to guide a medical device, for example a surgical 
tool, coupled to haptic device 113 toward a target area on anatomy 114 of the patient. The 
path of the haptic object may depend on a surgical plan. An algorithm may be used to create 
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the funnel shaped haptic object illustrated in FIGURE 4A. The information desired to create 
the funnel shaped haptic object may be based on a surgical plan. If desired, haptic object 26 
may move with haptic device 113. This allows guidance of the surgical tool toward the target 
area from the current position of haptic device 113. Thus, the surgical tool may be guided 
toward the target area from any position in proximity to anatomy 114. Furthermore, the 
surgical tool may be guided from a current pose to a desired pose. 

[00761 Haptic object 26 may be of any shape, for example, a line, a curve, a 

cylinder, a funnel, and/or the like. Haptic object 26 is, in the illustrated example, defined as a 
virtual pathway to facilitate interactive positioning of haptic device 113 and/or surgical tool 
112 coupled to haptic device 113 at a desired position. Haptic object 26 guides surgical tool 
112 coupled to haptic device 113 from an initial position and/or pose toward a target area 
and/or a desired pose relative to anatomy 1 14 of the patient. If desired, haptic object 26 may 
guide surgical tool 1 12 to the target area along a path or trajectory 28. The path or trajectory 
28 from the initial position to the target area may depend on the siwgical plan. The path may 
be of any shape, for example a straight line, a curve, a funnel, a cylinder, and/or the like. 
Based at least in part on haptic object 26, haptic forces are applied to haptic device 1 13 as the 
user moves the surgical tool or haptic device to guide the user in moving the surgical tool 112 
along path 28 toward the target area. 

[0077] Haptic object 26 is preferably steerable or reconfigurable. For 

example, the haptic object may be defined to move or to change position and/or orientation as 
the haptic device (or the surgical tool or instrument coupled to it) moves. This allows, for 
example, the user to guide surgical tool 1 12 toward the target area from almost any position 
in proximity to anatomy 114. This reconfigurability or steerability of haptic object 26 also 
allows the user to guide surgical tool 1 12 to the desired pose firom its current position and/or 
pose. 

[0078] Haptic object 26 may also be allowed to move from a pre-defined path 

or position in order to avoid obstacles, preferably without deviating from the target area. 
This is especially useful in avoiding obstacles in the path of haptic device 1 13 that computer- 
assisted surgery system 11 may not be aware of Thus, surgical tool 112 may be steered by 
the user toward the target area without colliding with other surgical tools and equipment, the 
patient, or operating room staff. 

[0079] Steering, moving or reconfiguring is, in a preferred embodiment, in 

response to application of a force or torque on the haptic device or the haptic object that 
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exceeds a threshold value. For example, if the user pushes haptic device 113 against the 
haptic object with a force that exceeds a threshold, then the haptic object will be repositioned, 
reconfigured or modified to a new configuration based on the input force or torque. 
Preferably, haptic object 26 moves in the direction of the force or torque thereby providing an 
intuitive method for repositioning or realigning haptic object 26. 

[0080] If desired, haptic object 26 may move to a new location if the target 

area is changed. Thus, as shown in FIGURE 4A, haptic object 26 may be moved from an 
initial position to a new position, as shown by haptic object 26', in response to a change in 
the target area. 

[0081] In an altemative embodiment, haptic object 26 may be defined as 

virtual linear or non-linear springs, dampers, clutches, and/or the like, logically applied to one 
or more joints of haptic device 113. One or more joints of haptic device 1 13 may comprise 
virtual detents corresponding to the final desired pose of haptic device 113. Preferably, 
standard joint-space control techniques are used to implement the haptic objects at each joint 
and conventional inverse kinematics techniques are used to determine the joint positions 
corresponding to the desired Cartesian position/angle of the haptic device. The user may 
avoid obstacles by specifying the sequence in which the joints of haptic device 113 "lock" 
into their detents. The user may be permitted to modify the selected sequence by "unlocking" 
joints during positioning of surgical tool 112, especially if the sequence is determined 
through a trial-and-error technique. Interactive unlocking of a joint by the user may be based 
on the magnitude, duration or dynamic property of the force and/or the torque at that joint by 
the user. A graphical user interface, a footswitch, a keyboard, a button, and/or the like, 
communicatively coupled to haptic device 113 may be used to imlock a joint. If desired, 
once the desired pose is achieved, the ability to unlock the joints may be disabled to prevent 
inadvertent motion of haptic device 113. 

[0082] In another altemative embodiment, haptic object 26 may be defined by 

virtual linear or non-linear springs, dampers, clutches, and/or the like, logically associated 
with one or more redundant degrees-of-freedom of haptic device 113. For example, if a 
haptic device comprising of four joints is used to position the tip of surgical tool 1 12, then the 
haptic device 113 may be moved along one of the degrees-of-freedom without affecting the 
position of the tip. Haptic object 26 may be associated with the redundant degree-of-fireedom 
to permit the user to interactively modify the position of haptic device 1 13. 
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[0083] FIGURE 4B is a flowchart of a method 170 for interactive haptic 

positioning of a medical device, for example surgical tool 1 12 mounted to haptic device 1 13, 
using a reconfigurable or steerable haptic object 26, all as shown in FIGURE 4A. If desired, 
the reconfigurability of the haptic object may be user-configurable such that the user may 
turn this feature ON or OFF depending on the application or depending on the step of a 
particular application. When the reconfiguration feature is enabled, method 170 is preferably 
executed periodically. 

[0084] In step 172, a determination is made as to whether the medical device 

is in a desired pose. This determination may be made by using sensing information from one 
or more position sensors, such as encoders or resolvers, which may be integrated in the haptic 
device. If desired, this determination may be made by using sensing information from an 
external device, such as a laser interferometer, a camera, and/or other tracking device. 

[0085] If in step 172, it is determined that the medical device is in the desired 

pose, then in step 174, haptic interaction forces and/or torques to maintain the pose of the 
medical device are determined. This determination may be made based at least in part on the 
position and/or velocity of the haptic device and/or the medical device relative to the desired 
pose. Any control algorithm now known or later developed may be used for this 
determination, for example, robust control, adaptive control, hybrid position/force control, 
Proportional-Derivative (PD) control, Proportional-Integral-Derivative (PID) control, 
Cartesian based control, inverse Jacobian control, transpose Jacobian control, and/or the like. 
The determined haptic interaction forces and/or torques may be transformed and provided to 
the haptic device. If in step 172, it is determined that the medical device is not in the desired 
pose, then in step 176, haptic interaction forces and/or torques to maintain the medical device 
within a haptic object are determined so that the medical device may be guided toward the 
target area. In step 178, a determination is made as to whether the result of at least one scalar 
valued function of the haptic interaction forces and/or torques calculated in step 176 exceeds 
at least one reconfiguration threshold. The reconfiguration threshold may be user- 
configurable. A scalar valued fiinction computes a value based on one or more input values. 
In an exemplary embodiment, the scalar valued fiinction may be the square root of the sum of 
the squares of the input values. A scalar valued fimction may be applied to one or more 
haptic interaction forces to provide a scalar value. The resulting scalar value may be 
compared to the reconfiguration threshold. Dynamic properties of the haptic interaction 
forces and/or torques, such as direction, duration, and/or the like, may also be considered. 
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[0086] If the result of none of the scalar valued functions exceeds the 

reconfiguration threshold, then the process ends. Otherwise in step 180, haptic object 26 is 
modified based at least in part on the haptic interaction forces and/or torques. For example, if 
the surgeon guides the haptic device such that the haptic device in effect pushes against the 
haptic object, the value of the scalar valued function of the haptic interaction forces and/or 
torques generated to keep the haptic device within the haptic object may exceed the 
reconfiguration threshold. In such a case, it is desirable that the haptic object be modified, 
for example in the direction of the force applied by the surgeon such that the surgical tool is 
maintained within the haptic object. The modification of the haptic object may comprise 
changing the size of the haptic object, changing the shape of the haptic object, pivoting the 
haptic object along the target area of the patient's anatomy, and/or the like. 

[0087] A technical advantage of this exemplary embodiment for interactive 

haptic positioning of a medical device is that by modifying a haptic object based on the haptic 
interaction forces and/or torques, greater flexibility is provided to the surgeon. Thus, the 
surgeon may approach the target area without colliding with other surgical tools and 
equipment, the patient or operating room staff, and still be provided with haptic cues to 
enable the surgeon to guide the surgical tool to the target area. 

[0088] The illustrated method for interactive positioning of a haptic device 

using a reconfigurable (repositionable, steerable) haptic object may be used in any situation 
where it is desirable to move the haptic device, optionally coupling a component of interest, 
such as a medical device, for example a surgical tool, and/or the like, within a cluttered or 
safety-critical environment. If desired, the haptic device itself may be the component of 
interest. The illustrated method may be used in a variety of applications, such as a procedure 
where virtual constraints and/or haptic cues are used to move the component of interest into a 
predefined location and/or orientation and safety or other concerns make autonomous device 
motions undesirable. For example, the method may be used in an implant placement 
procedure, a biopsy procedure, deposition of therapeutic implants, diagnostic palpation of 
intemal or external anatomy, tumor removal, radiation therapy, artistic or commercial 
sculpting, artistic or commercial painting, scientific or engineering experiments, such as 
surface digitizing, sample collection, circuit board probing, manual assembly, fabrication or 
testing of mechanical and/or electronic components or assemblies, material handling, and/or 
the like. 
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[00891 For rehabilitation and/or physical therapy applications, a haptic device 

may be coupled to the patient using an orthotic device, which may require the patient to grasp 
a handle. In such an embodiment, the haptic device may be coupled to a computer system 
having a user console. The computer system may or may not be a CAS system, but may be a 
computer system designed for rehabilitative or physical therapy applications. If desired, the 
computer system may be integrated with computer 10. The orthotic device may have straps, 
braces, shells, or cast features to provide a firm or loose connection as desired. The orthotic 
device allows the haptic device to guide, monitor, and/or assist rehabilitative motions or other 
exercises. For example, the patient or a therapist may couple the patient's arm or leg to the 
haptic device and lead it through a desired motion while the haptic device records the 
properties of the motion. The motion can then be repeated multiple times without the 
assistance of the therapist. The haptic device may also be used to monitor the patient's 
efforts to move by noticing how much effort is required to move the patient, or through the 
use of force sensing devices which may be coupled to the haptic device at or near the location 
where the patient interfaces with the haptic device. The haptic device may also be used to 
simply constrain the patient's motion to the defined path which requires the patient to 
advance along the defined path using their own strength. Modes where there is a shared 
effort between the patient and the haptic device may also be advantageous. It is desirable that 
when used in this manner, the haptic device operate in a safe manner because it is so close to 
the patient, who may have only partial function in one or more extremities. It may be 
undesirable for the haptic device to move to new positions automatically or autonomously. 
However, it may be desirable to reposition the haptic device, for example to permit initial 
attachment to or grasping by the patient, so that the haptic device may be moved to different 
starting positions between different exercises or repetitions of the same exercise, or in the 
course of performing the rehabilitative motions or exercises. A physical therapist may 
provide the interactive input for repositioning the haptic device. If desired, the patient may 
provide such input while interfacing with the haptic device. 

[0090] The illustrated method for interactive haptic positioning of a surgical 

tool using a reconfigurable or steerable haptic object may be implemented in software, 
hardware, or a combination of both software and hardware. The steps discussed herein need 
not be performed in the stated order. Several of the steps could be performed concurrently 
with each other. Furthermore, if desired, one or more of the above described steps may be 
optional or may be combined without departing from the scope of the present invention. 
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[0091] Referring now to FIGURE 5, when the user interacts with a haptic 

object, such as haptic object 20, it is sometimes desirable to know the magnitude of forces 
applied to the haptic object or the amount that a real or virtual tool or implant is penetrating 
the haptic object. For non-trivial haptic objects, or those with complicated two or three 
dimensional forms, it may be difficult to present this information in a manner that is simple 
for the user to understand. However, the desirable piece of information is often the local 
penetration distance or haptic repulsion force. While these can be up to three-dimensional 
vector quantities, the magnitude (or length) of such vectors, possibly in the direction of a 
local unit normal of the haptic object, are most useful for augmenting the haptic interaction of 
the user. These magnitudes are simple one-dimensional quantities and can be conveyed to 
the user in a variety of methods, including meters, dials, numerical displays, graphs, and 
other visual methods, but also with audio, tactile, haptic, or other means. 

[0092] Though a complete message is conveyed directly by haptic device 1 13 

to the hand of the surgeon, a visual or audible display can be used to support rich interactions 
between the user and the system. For example, well known and commercially available 
speech recognition techniques can be used to provide a verbal method for the user to 
communicate information or instructions to the computer aided surgery system. Speech 
output from the computer aided surgery system 11 can also be used for communicating 
information to the user including status information, warning messages, event notification, 
and responses to user queries, whether communicated verbally or through some other 
method. Computer monitors, projection displays, wearable displays, head-mounted displays, 
stereoscopic views, holographic displays, and/or other visual display devices can be used to 
provide schematic anatomic representations, images of diagnostic datasets, instructions or 
guides for the surgical procedure, depictions of virtual and haptic objects, system status 
information, patient information, and other information that is easily communicated over a 
visual display. Any other input or output device could similarly be used to augment the 
haptic interaction between the user and the computer surgery system. 

[0093] A visual and/or audio display of the penetration into a haptic object of 

a predetermined stiffness of a surgical device's depth, force and/or velocity is provided. The 
haptic object is based upon information from the computer-assisted surgical system. The 
display is one-dimensional in order to facilitate the commimication of the local penetration 
magnitude of the surgical device into the haptic object. 
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[0094] During surgery, the haptic device may be used to enhance the 

performance of the surgeon in, for example, such tasks as holding a tool steady, making 
straight cuts, or moving a tool tip along a path or surface. The haptic device can replace 
mechanical cutting jigs and alignment apparatus used for aiding in the placement of and 
preparation of anatomy for implanted medical devices. Virtual haptic surfaces may be used 
to replace physical cutting blocks. The virtual haptic surfaces in this instance are preferably 
software entities that can be easily and cheaply created from the models of the implant. The 
virtual haptic surfaces can be created with curved shapes, which more closely match the 
underlying anatomy and enable implant designs that require less bone or tissue removal. 

[0095] Sculpting of a physical object, such as a bone, frequently requires 

multiple planar features to be created in the bone and/or on the surface of the bone. A haptic 
object may be defined to assist in such sculpting. The shape of the defined haptic object may 
correspond substantially to the desired resulting shape of the physical object after sculpting. 
The physical object and the haptic object may have segments or surfaces with abrupt 
transitions and/or may have portions with short radius of curvature. As such, it is possible 
that a surgical tool coupled to the haptic device and being used to sculpt the physical object 
may abruptly fall off one segment causing unintentional damage to the physical object or 
other objects in the vicinity of the physical object, or be distracting or disturbing to the user. 
A segment may be one-dimensional, two-dimensional or three-dimensional. 

[0096] In order to address this problem, haptic object is dynamically modified 

during sculpting in order to prevent the surgical tool or the haptic device from following an 
abrupt transition from one segment of the haptic object to another segment. Preferably, the 
haptic object remains in the modified form only so long as it is desirable to prevent abrupt 
transitioning of the surgical tool or the haptic device from one segment to another. Once the 
cutting or portion thereof is complete, the haptic object may be returned to its original 
configuration, for example to its original shape, size, orientation, and/or the like. The 
modification of the haptic object may comprise creating another haptic segment that prevents 
the surgical tool from following an abrupt transition from one segment of the haptic object to 
another segment of the haptic object, modifying an existing segment of the haptic object, for 
example by extending the existing segment beyond its boundary, and/or the like. 

[0097] FIGURE 6A illustrates an exemplary haptic device being used for 

haptic sculpting of physical objects with high curvature. FIGURE 6B illustrates an 
exemplary haptic object 20 for haptic sculpting of physical objects with high curvature. 
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Referring now to FIGURES 6A and 6B, in certain cases, the desired shape for an anatomical 
region to be prepared with the aid of a haptic object may include sharp external edges. It is 
difficult to properiy execute cuts without slipping off these edges, resulting in rounding of the 
edges and other xmwanted artifacts in the resulting contour of the anatomy. An improved 
method for preparing these types of shapes involves dynamically enabling and disabling 
portions of the haptic surface. In particular, this method is helpful if a haptic object contains 
at least one sharp external edge where the local angle between the two portions joined by the 
edge as depicted in FIGURES 6 A and 6B is less than 180 degrees. The method includes a 
way of selecting one of these portions, which may include any of the user input modalities 
mentioned herein, but the preferred method is based on proximity to the haptic object. When 
one of the portions is selected, that portion of the haptic object is extended beyond the joining 
edge to provide a continuous guide surface. When the extension is no longer required, the 
user can return the haptic object to its original configuration by moving the haptic arm away 
from the portion or using any other input modality. 

[0098] For example, in a total or unicompartmental knee replacement 

procedure, multiple planar cuts are often required to prepare the femur for the femoral 
implant. A haptic object is defined in software that contains portions closely related to the 
desired femoral cuts. In experiments, when the user attempts to resect the bone using a 
cutting burr mounted in the haptic arm using the full haptic object, it is difficult to make the 
straight cuts without slipping from one portion to another and frequently moving the burr 
beyond the desired region. This slipping may result in damage to tendons, blood vessels, 
ligaments, and other structures and distract the user. If instead, each cutting plane of the 
haptic object is extended when the user brings the cutting burr within close proximity to that 
portion, it is much easier to create straight cuts without moving beyond the local anatomical 
site. The portion is returned to its original extent by simply moving back away from it, at 
which time the user can bring the cutting burr into contact with any of the other portions to 
extend them in a similar manner. While foot pedals, voice commands, or other input 
modalities can be used to control the extension of each plane, controlling them in the 
preferred manner described previously requires no additional hardware and is extremely 
simple for the user. However, a visual display of the haptic object and the extended portion 
is also helpful for helping the user to understand more complex haptic objects, especially 
where their view of the cutting is limited due to obstructions or a minimally-invasive 
technique. 
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[0099] FIGURE 6A shows an exemplary system for dynamically extending a 

haptic object. A representation of the physical object, for example the anatomy of the patient 
to be sculpted, may be displayed on display device 30. The representation of the physical 
object may comprise a two-dimensional or three-dimensional drawing or image. The image 
could be, for example, a two-dimensional medical diagnostic dataset or a three-dimensional 
medical diagnostic dataset of the patient. In FIGURE 6A, haptic object 20 includes two 
different portions (20' and 20") divided by a well defined edge 21. When haptic device 113, 
the surgical tool, or the virtual surgical tool comes within a predefined distance, say Ri, of 
one portion, say portion 20', that portion of haptic object 20 is activated. If desired, the 
activated portion of the haptic object 20 may be extended as shown by the broken lines 23 in 
FIGURE 6A. When haptic device 113 moves to within a predefined distance of another 
portion, say portion 20", the new portion of haptic object 20 may be activated. If desired, the 
newly activated portion of haptic object 20 may be extended. 

[00100] It is desirable that haptic object 20 with high curvature be logically 
divided into or be approximated by a plurality of portions or segments without high 
curvature. For example, as shown in FIGURE 6A, haptic object 20 may be logically divided 
into a plurality of portions 20' and 20" separated by an edge 21. Although, it is preferable to 
logically divide a haptic object into a plurality of segments, the haptic object itself may be 
defined using a logical combination of a plurality of segments. For example, a plurality of 
segments may be initially defined and the haptic object may be defined as a logical 
combination of one or more of the plurality of segments. If desired, the haptic object may 
comprise a regular or irregular arrangement of volume elements, or voxels, some or all of 
which may be labeled. It may be desirable to only label the voxels on the surface of the 
object in this manner. 

[00101] FIGURE 6C is a flowchart of a method 120 for dynamically modifying 
a haptic object, such as haptic object 20 of FIGURES 6A and 6B. If desired, the dynamic 
modification feature may be user-configurable such that the user may turn this feature ON or 
OFF depending on the application or depending on the step of a particular application. When 
the dynamic modification feature is enabled, method 120 is preferably executed periodically. 

[00102] In step 122, a determination is made as to whether a configuration of 

the haptic object, say haptic object 20, has already been modified, for example by modifying 
a segment of the haptic object or by adding a new segment. In the preferred embodiment, the 
value of a configuration flag may be checked to determine if haptic object 20 has already 
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been modified. If haptic object 20 has not already been modified, then in step 124, a 
determination is made as to whether one or more criteria for modifying the configuration of 
haptic object 20 is satisfied. The criteria may be proximity of surgical tool 112 coupled to 
haptic device 113 to haptic object 20, penetration of haptic object 20 by surgical tool 112, 
gestural motions of surgical tool 1 12, gestural or other motion of surgical tool 1 12 relative to 
the position of haptic object 20, a fixed or variable time period, detection of an unwanted 
slippage over edge 21, and/or the like. If desired, the criteria may be proximity of the 
representation of surgical tool 112 to haptic object 20, penetration of the boundaries of haptic 
object 20 by the representation of surgical tool 112, gestural or other motion of the 
representation of surgical tool 112 relative to the position of haptic object 20, and/or the like. 
When modification of the configuration of haptic object 20 comprises modifying a segment 
of haptic object 20, preferably the same criteria is used to determine if any of the segments 
should be modified. However, if desired, different segments may be modified based on 
different criteria. In such an embodiment, each of the plurality of segments may have one or 
more criteria associated with it. 

[00103] If in step 124, it is determined that at least one criteria for modifying 
the configuration of haptic object 20 is satisfied, then in step 126, the segment to be modified 
is selected. Altematively, a segment in proximity to which a new haptic segment is to be 
created may be selected in step 126. In an altemative embodiment, the process starting at 
step 126 may be executed if a predefined logical combination of a set of criteria are satisfied. 
Preferably, the segment that is closest to haptic device 113 is selected. However, if desired, 
other criteria may be used to select a segment. For example, if surgical tool 112 has crossed 
an edge between two or more segments since the last time method 120 was executed, then 
one of the segments associated with the edge that was crossed may be selected. 
Altematively, the segment being penetrated by surgical tool 112 may be selected. In step 
128, the configuration of the selected segment is modified, preferably by extending the 
selected segment in a desired direction of movement of haptic device 113. The configuration 
flag may be set to indicate that haptic object 20 has been modified. 

[00104] The method for modifying the configuration of the selected segment is 

preferably based at least in part on the manner in which the haptic object is represented. This 
representation may be based on surface polygons, voxels, non-uniform rational B-splines 
(NURBs), constructive solid geometry, and/or any other method for representing haptic 
objects now known or later developed. The modified segment may be represented in any 
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manner which may or may not be the same as those used to represent the original haptic 
object. Preferably, the selected segment is extended such that the extended portion is 
continuous with the segment along one of its high curvature edges. The extension may be 
flat or curved. The segment may be extended a fixed or variable distance beyond the original 
segment, or could be extended to intersect another portion of the haptic object or the edge of 
a workspace. The method used for extending the segment depends on the method used for 
representing the extension. For example, if a haptic object is represented with surface 
polygons, then the polygons that lie within the segment of interest and adjacent to one of its 
boimdaries are identified. A neighboring segment that lies beyond the original segment and 
has the same normal direction as the original polygon may be enabled. For a voxel 
representation, the voxels may be labeled to indicate whether they behave as solid or filled 
regions of space for configurations of the haptic object with different extended segments, 
which may be automatically, semi-automatically, or manually designed. The selected 
neighboring segment may be added to the haptic object. Thus, as illustrated in FIGURE 6A, 
if portion 20' of haptic object 20 is the selected segment, then portion 20' may be extended 
beyond its original boundary, for example as shown by broken lines 23. Alternatively, if 
desired, a new haptic segment may be created in proximity to the selected segment. 

100105] In step 130, haptic interaction forces and/or torques for the haptic 

object are calculated. The haptic interaction forces and/or torques may be transformed and 
provided to haptic device 113. For example, it may be desirable to compute appropriate 
forces and torques for the actuators of the haptic device to apply such that the desired haptic 
interaction forces and/or torques will be produced. In some cases, it may be desirable to alter 
position or velocity commands to the actuators to produce the desired effect. The haptic 
interaction forces and/or torques from the selected segment may be used to guide haptic 
device 113 in a desired direction away from, toward, or aligned with physical object 114 to 
be sculpted. The haptic interaction forces and/or torques may be repulsive, attractive, 
ftictional, viscous, impulsive, detent, regulatory (for example designed to maintain cutting 
speeds or feed rates), and/or the like. If desired, the haptic interaction forces and/or torques 
may be calculated using a mathematical, control theory, or machine learning algorithm. 

[00106] If in step 124, it is determined that the criteria for modifying the 

configuration of haptic object 20 is not satisfied, then the process starting at step 130 may be 
executed. 
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[00107] If in step 122, it is determined that the configuration of haptic object 20 
has already been modified, then in step 134, a determination is made as to whether one or 
more predefined criteria for maintaining haptic object 20 in the modified configuration is 
satisfied. These criteria may or may not be the same as those considered when the 
configuration of haptic object 20 was initially modified. Preferably, if at least one criterion 
for maintaining the haptic object in the modified configuration is satisfied, then the process 
starting at step 130 may be executed. Otherwise, in step 136, the modified haptic object is 
returned to its original configuration. The configuration flag may be reset to indicate that 
haptic object 20 has not been modified. After execution of step 136, the process starting at 
step 130 may be executed. In an alternative embodiment, the process starting at step 130 may 
be executed if in step 134 it is determined that a predefined logical combination of a set of 
criteria are satisfied. 

[00108] As illustrated in FIGURE 6A, when haptic device 1 13 or surgical tool 
112 coupled to haptic device 113 comes within a predefined distance Ri of one portion of 
haptic object 20, say portion 20', that portion of haptic object 20 may be activated and 
modified such that it extends beyond its original boimdary as shown by dashed lines 23. 
While haptic device 1 13 or surgical tool 1 12 is in close proximity to portion 20' or maintains 
contact with portion 20', portion 20' remains modified. Surgical tool 112 may be used 
during that time to sculpt the portion of physical object 1 14 corresponding to portion 20' to a 
desired shape. When the sculpting of the portion of physical object 114 corresponding to 
portion 20' is completed, the user may move haptic device 113 away firom portion 20'. 
Portion 20' may then be returned to its original configuration. When haptic device 113 or 
surgical tool 112 moves to within a predefined distance of another portion of haptic object 20, 
say portion 20", portion 20" of haptic object 20 may be activated and modified such that it 
extends beyond its original boundary. 

[00109] The illustrated method for dynamically modifying a haptic object may 
be used in a variety of applications, such as any procedure where a virtual constraint and/or 
haptic cues are used to guide a user using a haptic device for sculpting a physical object or 
shape that has high curvature. For example, the method may be used in fabrication of 
components for consumer or industrial products, for the reproduction or creation of artistic 
pieces, such as sculptures, for shaping bones in an orthopedic procedure, and/or the like. 

[00110] The illustrated method for dynamically modifying a haptic object may 
be implemented in software, hardware, or a combination of both software and hardware. The 
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steps discussed herein need not be performed in the stated order. Several of the steps could 
be performed concurrently with each other. Furthermore, if desired, one or more of the above 
described steps may be optional or may be combined without departing from the scope of the 
present invention. 

[00111] A technical advantage of this exemplary embodiment for dynamically 
modifying a haptic object is that the sculpting of the physical object may be performed in a 
more controlled manner. Thus, during a surgical procedure, unintentional damage to parts of 
the body may be avoided and the user can feel more comfortable using the system. Another 
technical advantage is that the user does not have to move its attention away from the 
working volume when switching from one segment to another segment of the haptic object. 
Yet another technical advantage is that shapes with high curvature may be operated on more 
easily than if only the entire haptic object were used. 

[00112] FIGURE 8 illustrates the use of an exemplary haptic device 113 as an 
input device. Haptic device 113 and a haptic object 20 in real space are illustrated. Haptic 
device 113 may also be used as an input device, allowing information to pass from the user to 
CAS system 11, and providing functionality similar to connmon user interface devices, such 
as a mouse, touchpad, keyboard, joystick, flight controller, haptic joystick, or any other input 
device. When used as an input device, it may be used for defining anatomical reference 
geometry, manipulating the position and/or orientation of virtual implants, manipulating the 
position and/or orientation of surgical approach trajectories, manipulating the position and/or 
orientation of bone resections, and the selection or placement of any other anatomical or 
surgical feature. Haptic device 113 may also be used for more generic user interface 
functions, including but not limited to, moving a cursor 3 1 (FIGURE 8), selecting buttons or 
other similar user interface objects, selecting pull-down menus, manipulating on-screen dials, 
knobs, and other controls. When in this user-input mode the haptic device can be constrained 
to move in only certain directions which may be defined relative to the position of a 
predetermined portion of the haptic device, relative to the position of the patient or a portion 
of the patient anatomy, or relative to images or 3-D models of schematic, virtual, atlas, or 
actual patient anatomical features. The predetermined portion of the haptic device may be 
capable of moving. 

[00113] As illustrated in display 30 of FIGURE 8, haptic device 113 may be 
used as an input device to change the position, shape, size, etc. of haptic object 20. An 
example of an application of haptic device 1 13 used in this mode is planning the placement of 
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a knee implant. After acquiring appropriate anatomical images of the anatomy of interest, the 
computer surgery system enters a mode where a cursor appears on a display visible to the 
user. The user grasps the arm to move the position of the cursor, possibly in multiple views. 
When satisfied with the position of the cursor, the user fixes it in the desired position through 
the use of a foot pedal, button, wired or wireless control pendant, voice command, or other 
input, or through the application of a force or torque to the haptic arm, or moving the haptic 
arm in a distinctive gesture, such as a tap, twist, or other gesture that is easily distinguishable 
from the user interactions during the cursor positioning. After the first position is set, a 
second cursor is used to define the endpoint of a line connecting to the two or three- 
dimensional position of the first cursor. The second cursor is moved, as above, to define an 
anatomical axis of the femur bone and its position is fixed using one of the above mentioned 
techniques. The two or three dimensional position and orientation of the implant can then be 
manipulated by the user using the haptic device as an input device. The implant is 
constrained by the system such that one of its surfaces is perpendicular to the anatomical 
reference line, but its position and orientation can be adjusted by the user. It is also possible 
to allow deviations from the anatomical axis, possibly coupled with displays of such 
deviations relative to anatomical reference frames familiar to the user. For example, the 
varus/valgus angle of the implant relative to the anatomical reference line can be adjusted and 
displayed to allow appropriate alignment of the knee implants. This general technique can be 
adapted to plan the approach and/or placement of minimally invasive hip and knee implants, 
trauma fixation pins, pedicle screws, biopsy needles, radioactive beads, radiotherapy beam 
emitter, or any other medical device. 

[00114] With a haptic device, the surgeon can use tools identical or very 
similar to those used in standard practice. By exploiting the haptic features of the device, the 
need for awkward teach pendants or GUI-based robot controls may be reduced or eliminated. 
Switching between freehand and assisted steps of a procedure is quickly performed by simply 
pushing the device out of the way, similar to familiar operating room objects such as 
microscopes and overhead lights. While the systems may be internally complex, the surgeon 
must be shielded from this complexity so that he can focus all of his attention on his patient. 

[00115] For example, the haptic arm can hold itself at a reference position 

using a joint-based or Cartesian control algorithm. The user applies forces and/or torques to 
the arm, either on an interaction handle or end-efifector or at any point on the arm, which 
cause the arm to deflect from the reference position. The amount and direction of the 
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deflection is continuously communicated to the computer system to modify the position of 
any desired virtual reference geometric feature or user interface object. 

[00116] In another example, the haptic arm can hold itself at a reference 
position using a joint-based or Cartesian control algorithm but with two degrees of freedom 
left unconstrained. The user can then move the arm in the unconstrained directions to 
provide two-dimensional control of a user-interface object, such as a cursor, implant, or other 
geometric or virtual surface entity. A similar technique can be used for one degree of 
freedom manipulation of objects, such as user interface slider bars, implant lengths, positions 
of objects along a reference trajectory, or any other one-dimensional control such as audio 
volxmie, image brightness, object scaling, image zooming, and the like. A similar technique 
can be used for higher than three degree of freedom positioning of implants or virtual or 
haptic objects. The haptic object's position may also be constrained relative to any relevant 
anatomical features for a particular application. For example, a knee implant may be 
constrained to have the proper alignment relative to the anatomical axis of the leg, or to 
achieve proper ligament balance, but with the other degrees of freedom controllable by the 
user in the manner described above. 

[00117] The stiffness or damping of the control algorithm may vary in different 

directions to indicate preferential directions of motion which may be aligned with any 
direction as described in the previous paragraph. This stiffness variation may include zero 
stiffness along certain directions or may lock the user to the preferred directions once the 
deviation from the reference position exceeds some threshold value. This stiffness variation 
assists with simplifying the planning process by allowing the user to focus their attention on a 
limited number of degrees of freedom at a time. For example, the user may set the position 
of an implant along one or two directions first, then set the position of the implant along an 
additional direction or directions without disturbing the set directions. 

[00118] The stiffness and damping variations can occur automatically 
depending on the physical interaction of the user with the haptic device and does not require 
the use of another input device such as a voice command, control pendant, or foot pedal. Any 
such simplification has benefits in reducing service costs, simplified system use, and 
improved safety. This general method of planning also allows the surgeon to perform 
planning without having to leave the normal operating position to interact with the computer- 
aided surgery system or requiring an assistant to control the computer-aided surgery system 
or requiring the introduction of additional input devices other than the haptic device which is 
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already being used for execution of the surgical plan. An additional benefit of this use of a 
haptic device is that the motion of the controlled object can be scaled relative to the motion of 
the arm, so that it can be positioned to a precision better than the user can position a real 
object, eliminating the deleterious effects of the user's hand tremor and any force 
disturbances arising from friction, backlash, magnetic detent forces, and other force 
disturbances arising from the haptic arm. It should be noted that the primary function of the 
object controlled by the haptic device is something other than monitoring the pose of the 
haptic device or monitoring the pose of a component of interest that may or may not be 
coupled to the haptic device. 

[00119] FIGURES 7 A and 7B illustrate the use of a haptic device and a 
surgical tool to define a haptic object. In the illustrated example, haptic device 113 is being 
used as an input device to define haptic object 182. In order to use haptic device 1 13 as an 
input device to define haptic object 182, the user grasps surgical tool 112 coupled to haptic 
device 113. If desired, the user may grasp haptic device 113 itself. Using surgical tool 112 
the user traces the boundaries of a desired region, for example a portion of the anatomy with 
respect to which the haptic object is to be defined. The user may trace the boundary, for 
example by touching the end of surgical tool 112 to portions of the desired region of the 
anatomy. The motion of surgical tool 112 may be recorded and the locations of the endpoints 
traced by the user computed. The geometry and/or location of haptic object 182 may be 
determined based at least in part on the location of the endpoints. A haptic device creation 
mode may be used to specify the desired shape of the haptic object. For example, to create a 
cylindrical haptic object that corresponds to a resected portion 184 of anatomy 114, the user 
can trace a plurality of points on the boundary of resected portion 184. An appropriate 
cylindrical haptic object may be created using any technique now known or later developed. 

[00120] Material and other properties of the anatomy may be defined by 
probing the anatomy. For example, surgical tool 112 may include a force measurement 
device coupled to the tip of surgical tool 112. Alternatively, if desired, instead of surgical 
tool 112, a probe comprising a force measurement device may be coupled to haptic device 

113. When the user interfaces the force measurement device against a portion of anatomy 

114, the force may be measured by the force measurement device. The measured force may 
be displayed as a function of the distance the anatomy moves, if any, upon application of the 
force. The stif&iess of that portion of anatomy 1 14 may be calculated as the ratio of the force 
to the distance. If desired, haptic device 113 itself may be interfaced with a portion of 
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anatomy 114 and the force determined based on the torques provided by the actuators. In 
such an embodiment, haptic device 113 may make small or large movements or press against 
portions of anatomy 114 in an autonomous mode without any physical assistance from the 
user. The force may be determined using any Jacobian method now known or later 
developed. The graphical representation 186 of FIGURE 7B illustrates the force with which 
surgical tool 112 comes in contact with anatomy 114 as a function of displacement of 
anatomy 114. 

[00121] If desired, other types of sensing devices may be coupled to haptic 

device 113 or surgical tool 112 to determine other properties of anatomy 114. These 
properties may be used to determine the type of tissue that is in proximity to haptic device 
113. Thus, haptic device 113 may be used to differentiate between hard and sofl bones, 
healthy and diseases tissues, different types of healthy tissues, boundaries of anatomical 
structures, etc. Based on information received from haptic device 1 13, the type of the tissue 
may be automatically determined by CAS system 1 1 and displayed on display device 30. 

[00122] FIGURE 9 is a flowchart of a representative method 190 for using 
haptic device 1 13 as an input device. In step 192, the input mode is initiated. The user may 
initiate the input mode by any mechanism now known or later developed. For example, the 
user may use a graphical user interface, a footswitch, a keyboard, a button, and/or the like, to 
indicate that the user desires to use haptic device 113 as an input device. Haptic device 113 
may control a plurality of objects. However, it is desirable that it only control a single object 
at one time. As such, in step 194, an identification of an object to be controlled is received. 
The controlled object may be a cursor, a button, an on-screen dial, a knob, a slider bar, or 
other similar user interface object, a virtual implant, a surgical approach trajectory, a bone 
resection, and/or the like. The user may select the object to be controlled by any method now 
known or later developed, for example by selecting the object using a conventional input 
device. 

[00123] In step 196, a reference pose for haptic device 113 may be stored. The 

reference pose is preferably the current pose of haptic device 113. For example, in this step, 
position information about the tip of haptic device 113 may be stored. In step 198, the 
controlled object is correlated with haptic device 113. The correlation of the controlled 
object with haptic device 113 is desirable so that movement of haptic device 113 may be 
translated or mapped into a corresponding movement or action relative to the controlled 
object. The correlation or mapping allows a determination of the amount or direction of 
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movement of the controlled object in response to movement of haptic device 113. For 
example, the user may specify that movement of haptic device 1 13 by one imit should cause a 
controlled object, for example cursor 31, to move by ten pixels on display device 30. 

[00124] The user may move haptic device 113 aroxmd to control the object 
selected in step 194. In step 200, a change in pose of haptic device 113 is determined. The 
change in pose of haptic device 1 13 is preferably determined relative to the reference pose of 
haptic device 113. The change in pose of haptic device 113 may comprise, for example, a 
change in position of the tip of haptic device 113. 

[00125] In step 202, the reference pose of haptic device 113 may be updated. 
Preferably, the reference pose is updated based at least in part on the change in pose of haptic 
device 113. If desired, the reference pose may be updated based at least in part on a wrench 
applied to haptic device by the user. The wrench may be explicitly measured by a sensor. If 
desired, the wrench may be implicit in that the haptic device can determine that a wrench is 
being applied. 

[00126] In step 204, new parameters for the controlled object are calculated. 
The parameters of the controlled object may be, for example its pose, position, angle, size, 
color, shape, orientation, view direction, brightness, contrast, table indices, status, mode, 
configuration, and/or the like. The new parameters may be calculated based on the change in 
pose of haptic device 113 and/or the wrench applied to haptic device by the user. If desired, 
the new parameters may be calculated based on the change in reference pose of haptic device 
113. Preferably, correlation information obtained in step 198 is used to calculate the new 
parameters. The new parameters may be used to change the controlled object. Thus, for 
example, when the controlled object is cursor 3 1 and there is a change in pose of haptic 
device 113, then a new pose for the controlled object may be determined based on the new 
parameters. In step 206, the controlled object is changed based on the new parameters. Thus, 
for example, if the controlled object is cursor 31, then the position of cursor 31 on display 
device 30 may be changed based at least in part on the new parameters calculated in step 204. 

[00127] In step 208, a haptic wrench applied by the haptic device to the 

medical device and/or the user is determined. The haptic wrench may be determined based 
on the new parameters of the controlled object, the change in pose of haptic device 113, 
and/or the current pose of haptic device 113. 

[00128] In step 210, the determined haptic wrench is applied to haptic device 
113. Instead of allowing haptic device 1 13 to be moved in any direction, it may be desirable 
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to constrain the movement of haptic device 113. The detemiined haptic wrench when applied 
to haptic device 113 prevents it from moving in certain undesirable directions. For example, 
if the controlled object is capable of moving in only one dimension, it may be desirable to 
constrain the motion of haptic device 113 so that haptic device 113 moves in only one 
direction. As another example, when the object being controlled is cursor 31 on display 
device 30, then it may be desirable to constrain the movement of haptic device 1 13 to a two- 
dimensional plane corresponding to display device 30. As a further example, if it is not 
desirable for haptic device 113 to move large distances from the reference pose, the haptic 
wrench may act to retum haptic device 1 13 to the reference pose in one or more directions. 

[00129] Within the input mode, haptic device 113 may be used in a position 
control mode or a rate control mode. In the position control mode, the change in pose of the 
controlled object tracks the change in pose of haptic device 113. For example, if haptic 
device 113 is moved in a particular direction by one unit, the controlled object moves in a 
corresponding direction by a corresponding amoimt. When haptic device 113 is released, it 
stays in its new pose. 

[00130] On the other hand, in the rate control mode, the displacement of haptic 

device 113 from the reference pose and/or the wrench applied to the haptic device by the 
user, may control the velocity of the controlled object. For example, if haptic device 113 is 
maintained in its reference pose (or if no wrench is applied to the haptic device by the user), 
then the rate of movement of the controlled object is zero. The displacement of haptic device 
113 from the reference pose (or the magnitude of the wrench applied by the user to the haptic 
device) determines the velocity of movement of the controlled object with the velocity of 
movement being proportional to the displacement of the controlled object (or to the 
magnitude of the wrench applied to the haptic device). When it is desirable to move the 
controlled object, haptic device 113 is simply moved (or pushed) in the direction of the 
desired motion of the controlled object. When haptic device 1 13 is released it moves back to 
the reference pose due to application, in step 210, of the haptic wrench determined in step 
208. Thus, in the rate control mode, the controlled object may be moved a substantial 
distance without substantially moving haptic device 113. 

[00131] In step 212, a determination is made as to whether haptic device 1 13 is 
still operating in the input mode. If haptic device 1 13 is not operating in the input mode, then 
the process terminates. Otherwise, in step 214, a determination is made as to whether a new 
object to be controlled has been specified. If a new object to be controlled has not been 
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specified then the process starting at step 200 to determine the change in pose of haptic 
device 113 may be executed. Otherwise, the process starting at step 194 to receive 
identification of the new object to be controlled is executed. 

[00132] For example, in one embodiment, the reference pose may be associated 
with the desired trajectory of a drill guide attached to haptic device 113. In such an 
embodiment, updating the reference pose in step 202 comprises changing the desired 
trajectory of the drill guide. When the user moves haptic device 113 from the reference pose 
for a prolonged period of time, the reference pose will be updated to move in the direction of 
the user's deflection. If, in step 210, an appropriate haptic feedback wrench is applied, then 
upon release of haptic device 113 by the user, haptic device 113 will assimie the new 
reference pose. When the user is satisfied with the reference pose and the input mode is 
terminated in step 212, haptic device 113 will be in a pose such that the drill guide is aligned 
with the desired trajectory. 

[00133] The illustrated method for using a haptic device as an input device may 
be implemented in software, hardware, or a combination of both software and hardware. The 
steps discussed herein need not be performed in the stated order. Several of the steps could 
be performed concurrently with each other. Furthermore, if desired, one or more of the above 
described steps may be optional or may be combined without departing from the scope of the 
present invention. 

[00134] A technical advantage of using a haptic device as an input device in the 

manner described above is that the use of an additional input device may be avoided thereby 
reducing the clutter in the operating room. 

[00135] FIGURE 10 illustrates a system for conveying scalar information 
during a medical, surgical or interventional procedure. Haptic device 113 may be deployed 
as a fully integrated component of CAS system 1 1 or act as an optional peripheral to such a 
system. Location or position information of tool 112 coupled to haptic device 113 may be 
sensed and provided back to CAS system 1 1 with or without the use of sensor(s) 14. 

[00136] Medical, surgical and interventional procedures will be referred to 

collectively as "medical procedures." The medical procedure may involve resecting a portion 
of an anatomy, such as for a joint replacement, joint resurfacing, tumor removal, bone 
deformity correction and/or the like. If desired, the medical procedure may involve applying 
a synthetic, biologic, or therapeutic substance to a surface or region of interest, or placing a 
sensor, probe, implant or radioactive material in a desired position, surface or volume. When 
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a user interacts, for example, with a haptic object, it is sometimes desirable to know the 
magnitude of forces applied to the haptic object or the amount that a real or virtual tool or 
implant is penetrating the haptic object. For non-trivial haptic objects, or those with 
complicated two or three dimensional forms, it may be difficult to present this information in 
a manner that is simple for the user to understand. The desirable piece of information is often 
the local distance to a surface of interest or to a desired position, the local penetration 
distance of the surface of interest, or haptic repulsion force. While these can be up to three- 
dimensional vector quantities, the magnitude (or length) of such vectors, possibly in the 
direction of a local unit normal to the haptic object, are most useful for augmenting the haptic 
interaction of the user. These magnitudes are simple one-dimensional quantities and can be 
conveyed to the user in a variety of methods, including meters, dials, numerical displays, 
graphs, and other visual methods, but also with audio, tactile, haptic, or other means. 

[00137] In the exemplary embodiment of FIGURE 10, CAS system 11 is 

communicatively coupled to an audio source 216, for example a speaker, and display device 
30. During an exemplary medical procedure, anatomy 114, which in the embodiment of 
FIGURE 10 is a bone, is to be cut along haptic object 20 using the surface of haptic object 20 
as a guide surface. During the medical procedure, one-dimensional information, for example 
information regarding the scalar distance D of the tip of tool 112 from the surface of an 
object of interest, for example haptic object 20, is provided to the user, preferably but not 
necessarily automatically. CAS system 11 may be programmed, for example, to 
automatically provide this scalar information based on the stage of the procedure, the type of 
tool being used, the tool's position or proximity to an object (including surface) of interest, or 
other cues that assist CAS system 11 in identifying the object with respect to which 
magnitude information will be determined and displayed. 

[00138] In an exemplary embodiment, prior to initiation of the cutting 

operation, the value of D is positive. A value of zero for D indicates that the tip of cutting 
tool 112 is at the desired depth inside anatomy 114. In the exemplary embodiment, the 
desired depth is at the surface of haptic object 20. A negative value for D indicates that the 
depth of the tip of cutting tool 112 inside the bone is more than that desired. One- 
dimensional information may be communicated to the surgeon by any of a variety of 
methods, such as visual, audio, tactile, haptic, and/or the like. For example, a visual indicator 
218, for example a level meter, a dial, numerical displays, graphs, etc., may be used to 
indicate the value of D on display device 30 or any other device in proximity to the user. If 
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desired, audio source 216, visual indicator 218 and/or display device 30 may be provided 
closer to the user, for example on tool 1 12, haptic arm 113, other handheld tools, instruments 
or accessories, or wearable visual, audio, haptic, or tactile displays. For example, tool 112 
may include a simple display device or a multi-colored indicator, for example a multi-colored 
LED indicator, a multi-colored lamp indicator, a LED level meter, and/or the like, to indicate 
the cutting depth to the user. In such an embodiment, the user does not need to take their 
attention away from the surgical working area. Similarly, tool 112 may include an audio 
source to indicate the cutting depth to the user. In such an embodiment, the audio indication 
from audio source 216 may be easily heard by the user because of its proximity to the user. 

[00139] If desired, audio indicators or tones from speaker 216 may be provided 
instead of or in addition to visual indicator 218. For example, a series of beeps may be 
provided to indicate the value of D. As the value of D decreases, the interval between the 
beeps may be reduced correspondingly. If desired, the beeps may turn into a buzzing or any 
other sound when the value of D becomes zero and into a different sound, for example, a high 
pitched sound when the value of D becomes negative. In an exemplary embodiment, a 
positive value for D is an acceptable value and a negative value for D is an unacceptable 
value. 

[00140] An advantage of providing an audio indicator is that the surgeon does 

not have to take his/her eyes off the patient's anatomy, such as bone 114. However, if the 
noise in the operating room makes it difficult for the surgeon to hear an audio indicator, then 
a visual indicator may be more appropriate. 

[00141] If a haptic system or device is used, such as shown in the illustrated 
exemplary embodiment, a tactile indication may be provided through haptic arm 113 or 
through an auxiliary device. In an embodiment, haptic arm 113 vibrates to provide tactile 
indication to the user. The frequency, amplitude, waveform, and/or other property of 
vibration of haptic arm 113 is dependent on distance D. In another embodiment, a vibrating 
device may be provided on the user's body. The frequency, amplitude, waveform, and/or 
other property of vibration of the vibrating device is dependent on distance D. An advantage 
of providing a tactile indication is that the surgeon does not have to take his/her eyes off bone 
114. 

[00142] Because the one-dimensional information is easy to communicate to 
the user, the user is able to focus his attention on the task of cutting knowing that an audio 
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indication, a tactile indication or a quick glance at visual indicator 218 will inform him if he 
is cutting to the appropriate depth. 

[00143] Depth information may also be displayed to the user in situations 
where the user is not using a haptic device. For example, the user may be cutting the bone 
freehand with a tool whose pose is tracked by a tracking system. In such an embodiment, the 
user does not have the benefit of the haptic feedback that will tend to apply forces to the tool 
that keep it on the surface of a haptic object or prevent it from penetrating a haptic object. 
Instead of a haptic object, a simple geometric object, e.g., a curve, a point, line, a surface or a 
volume, may be used as the object of interest. The object of interest may be identified to the 
CAS system or the CAS system may determine it based on other information it has. For 
example, the object of interest may be defined directly with respect to the patient or with 
respect to a diagnostic image of the patient, or it can be derived or determined by the CAS 
system from other anatomical points or features identifiable by a user. The CAS system 
knows the position of the tool relative to the object of interest and displays one-dimensional 
depth information. In such a system, D may be calculated by determining the distance from 
the tip of tool 112 to the desired depth of cut of the object of interest. If desired, the object of 
interest may comprise a curve, a point, a surface, a line, a volume, or a set of desired 
positions. The object of interest may be a haptic object, a geometric object and/or the desired 
shape of the portion of the anatomy. 

[00144] If desired, in an alternative embodiment, the one dimensional depth 

display may be augmented with two-dimensional representations of the cutting depth at each 
point on the surface or three-dimensional representations of each point in space. The two- 
dimensional or three-dimensional representation may be a schematic or realistic depiction of 
the surface of interest. A property, for example color, brightness, and/or the like, of points of 
the surface of interest may be based at least in part on the distance between the tool and the 
respective points when the tool was closest to the respective points during the medical 
procedure. For example, different colors may be used to denote the distance between the tool 
and the points. The position of the tool when it was closest to a point during the medical 
procedure is determined and the distance between the tool and the point calculated. The color 
of the point reflects the distance from the tool to the point when it was closest to that point. 

[00145] FIGURE 1 1 is a flowchart of a method 220 for conveying depth 

information during a medical procedure. Method 220 is preferably executed periodically. In 
step 222, the distance of tool 112 from the desired surface is determined. Preferably, the 
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distance D of the tip of tool 112 from the surface of an object, such as haptic object 20, is 
determined. The process for determining a value for D depends upon how the object is 
represented intemally. This representation of the object may be based on surface polygons, 
voxels, NURBs, constructive solid geometry, and/or any other method for representing 
geometrical objects now known or later developed. The distance from the current location of 
tool 112 to an appropriate point on the surface of haptic object 20 is calculated. The 
calculated distance is assigned a positive or negative value depending on the relative position 
of the tip of tool 112 and bone 114 with respect to the surface of haptic object 20. In the 
example illustrated in FIGURE 10, if the tip of tool 112 and bone 1 14 are on opposite sides 
of the surface of haptic object 20, then D is assigned a positive value. Otherwise, D is 
assigned a negative value. 

[00146] In step 224, the determined distance is mapped to a desired output 
format for display to the user. For example, the distance may be mapped to an appropriate 
color, audio frequency, time period, sound, image, haptic cue, and/or the like. Preferably, the 
mapping is performed based at least in part on the value of D. Table A below shows an 
exemplary mapping table for a system where visual signals are provided to the user. 



D (Millimeter) 


OUTPUT FORMAT 


1-2 


Green light 


0.1-0.99 


Yellow Light 


0.0-0.09 


Red Light 


<0.0 


Black Light 


Tab 


leA 



[00147] In step 226, the determined distance is conveyed to the user in the 

desired output format. If the desired output format is a visual indicator, then in an exemplary 
embodiment an indicator of the appropriate color is displayed. For example, as specified in 
exemplary Table A, if the value of D is within an acceptable range, say between 1 and 2 
millimeters, then a green indicator is displayed, if the value of D is between 0.1 and 0.99 
millimeters, then a yellow indicator is displayed, if the value of D is between 0.0 and 0.09 
millimeters, then a red indicator is displayed and if the value of D is in an unacceptable 
range, say less than zero, then a black indicator is displayed. In the embodiment of FIGURE 
10, information about distance D is displayed by level meter 218. As the value of D changes, 
the color of a level in level meter 218 is changed. If the desired output format is an audio 
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indicator, then in an exemplary embodiment, distance information is conveyed by beeps as 
discussed herein with reference to FIGURE 10. 

[00148] A technical advantage of an exemplary embodiment of the present 
invention is that during a medical procedure information about the depth of a tool may be 
provided to the user in a simple manner so that the user may focus his/her attention on the 
medical procedure. 



